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Existing literature on the impact of Common Core State Standards in Math has shown little benefit, but it has not examined 
variation in outcomes based on implementation strategies, student subgroups, or outcomes other than test scores. We use a 
difference-in-differences approach with school fixed effects to compare outcomes in pre- and poststandards years across 
schools with different levels of participation in professional learning around the standards in the middle grades in Chicago. 
Postimplementation, there were significantly greater improvements in student reports of standards-aligned instructional 
practices, math grades, pass rates, and test scores in schools with more extensive professional learning around the standards, 
among students with low and average initial achievement. Relationships were largely not significant for students with high 
initial achievement. We discuss why Chicago might have seen positive results, including the district emphasis on professional 
learning around the practice standards and differential impacts based on student prior achievement. 
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THE adoption of the Common Core State Standards in 
Mathematics (CCSS-M) by most states across the country 
brought ambitious new targets for what students should 
learn and be able to demonstrate in school, and significant 
new demands on teachers. Almost 10 years after states began 
implementing the CCSS-M, there is only limited research on 
how student outcomes have changed in response to the new 
standards. The studies that exist focus exclusively on test 
scores and do not differentiate effects for students with dif- 
ferent achievement levels. They also tell us little about which 
strategies for implementing the standards might be effective, 
as they examine average changes across entire states. 

New standards by themselves cannot raise student 
achievement—they depend on the work of districts, schools, 
and teachers to interpret the standards and effectively shift 
instructional practice (Toch, 1991; Tyack & Cuban, 1995). 
Because the CCSS-M were intentionally constructed to 
allow flexibility in implementation across districts and 
schools, we can expect that the extent to which new stan- 
dards led to improved student achievement varied widely 
depending on district implementation strategies. After 
reviewing the literature on CCSS-M implementation, we 
identify teacher professional learning (PL) around instruc- 
tional practices as a particularly promising standards imple- 
mentation strategy. We then examine changes in student 
outcomes across schools in Chicago based on their use of 


supports for PL around the standards, asking the following 
questions: 


1. How did students’ reports of instructional practices, 
their test scores, grades, and pass rates in math 
change with implementation of the CCSS-M, based 
on their schools’ level of participation in standards- 
related PL? 

2. How did changes in math outcomes differ for students 
with low, average, and high prior math achievement? 


The CCSS-M Transition: Research on Implementation and 
Student Outcomes 


The CCSS-M contain two different types of standards. 
Content standards describe what students should learn at 
each grade level, whereas practice standards call for learn- 
ing “processes and proficiencies” to support conceptual 
understanding across content areas. The practice standards 
include asking students to “make sense of problems and 
persevere in solving them,” “reason abstractly and quantita- 
tively,” and “construct viable arguments and critique the 
reasoning of others” (Common Core State Standards 
Initiative, 2020c). Conceptual understanding, widely under- 
stood to derive from knowledge of key mathematical prin- 
ciples and how they relate to one another (Hiebert & 
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Lefevre, 1986; Rittle-Johnson et al., 2001), can be fostered 
by having students engage in these types of practices 
(Camburn & Han, 2011; Hattie, 2009; Prawat, 1989; Rittle- 
Johnson & Schneider, 2015). Integrated with instruction on 
procedural learning, they have been shown to improve stu- 
dent achievement (Baroody, 2003; Fennema et al., 1996; 
Kazemi, 1998; Newmann et al., 2001; Rittle-Johnson et al., 
2001; Wenglinsky, 2002). 

Research on whether the CCSS-M have actually led to 
improvements in math achievement is sparse and filled with 
caveats. Polikoff (2017) identified four priority areas for 
research around the CCSS-M: (1) studies on the effects of 
the standards on student outcomes, (2) research on imple- 
mentation, (3) research on how supports provided through 
district and school implementation plans change teacher 
practice and student outcomes, and (4) examination of het- 
erogeneous effects on subgroups of students. To date, there 
are only a few studies in each of these areas, and many gaps 
remain in what is known. 


Studies on the Effects of Standards on Student Outcomes 


Because the standards were adopted by almost all states at 
the same time, it is difficult to find comparison groups to 
develop strong causal estimates of their effects on students. 
Several studies have used states as the unit of analysis, com- 
paring student achievement in years before and after stan- 
dards implementation. One study compared states that were 
early adopters with late-adopting states, among states with 
low-rigor standards prior to the Common Core, and it found 
moderate effects on fourth-grade math scores (around 0.10 
SD), only modest effects on eighth-grade math scores (around 
0.04 SD), and no effects among economically disadvantaged 
students (Bleiberg, 2020). Another study found small 
improvements in ACT (American College Test) scores in 
Kentucky (0.03-0.04 SD), comparing cohorts of students 
exposed to the standards with earlier cohorts, but it was 
unable to conclude whether the impacts were due to the stan- 
dards or other changes in the state (Xu & Cepa, 2018). A 
series of studies compared changes in CCSS adopter states 
with strong implementation with a very small number (four 
to eight) of nonadopter states and found a mix of null, very 
small positive, and very small negative effects on student 
achievement, based on different specifications (Loveless 
2014, 2016). A further study compared states based on the 
degree to which the new standards differed from their previ- 
ous standards and found slightly negative long-term impacts 
of the CCSS-M that grew more negative over time (—0.10 to 
—0.06 SD; Song et al., 2019). One further study did consider 
local variation, comparing districts within the same state, 
based on their adoption of CCSS-aligned textbooks. It found 
that implementing districts had a slightly higher percentage 
of middle school students scoring proficient on the state 
achievement test compared with nonimplementing districts, 


while differences in math achievement across elementary 
students were insignificant (Gao & Lafortune, 2019). 

Thus, when examining changes in achievement, the find- 
ings are not consistent, the existing research typically exam- 
ines statewide averages without considering variation across 
schools and districts, and it often focuses on changes in the 
content standards. Moreover, existing research uses stan- 
dardized test scores as the sole measure of achievement. 
State tests may not be adequate for measuring student learn- 
ing from standards-aligned instruction; Polikoff and Porter 
(2014) found surprisingly small associations between stan- 
dards-aligned instruction and student test scores. Besides 
looking at test scores, research has yet to consider whether 
students’ course grades changed along with their probability 
of passing math. Students’ grades in their courses have been 
shown to be stronger predictors than test scores of high 
school success, graduation, college enrollment, and college 
persistence (Allensworth & Clark, 2020; Easton et al., 2017). 
Course failures are critical math outcomes, given the dem- 
onstrated risk of increased failure rates in response to higher 
academic demand (Allensworth et al., 2009; Clotfelter et al., 
2015; Simzar et al., 2016). 


Research on Standards Implementation 


At the outset of CCSS-M implementation, administrators 
expressed concerns about teacher preparation to teach math 
in an entirely new way (McLaughlin et al., 2014; Rentner & 
Kober, 2014). Studies of CCSS-M implementation across 
standards-adopting states have found that teachers report 
making significant changes to their instruction in response to 
the standards (Bay-Williams et al., 2016; Kane et al., 2016; 
Opfer et al., 2016). However, studies have also shown con- 
siderable variation in the extent to which teachers have 
changed their instruction (Edgerton & Desimone, 2018), 
with those who teach higher proportions of low-achieving 
students spending less instructional time engaged in stan- 
dards-aligned practices than others (Schweig, Kaufman, & 
Opfer, 2020). Whether students themselves perceive differ- 
ences in the types of instructional tasks they engage in 
during their math classes has not yet been studied, and 
instructional changes may be perceived differently for dif- 
ferent subgroups of students. 


Research on the Effects of Implementation Supports on 
Student Outcomes 


While the CCSS-M were adopted by states, how to imple- 
ment them in schools has been largely left up to districts 
(Desimone et al., 2019). Districts could support instructional 
change through many different mechanisms, including 
adopting new textbooks, digital tools, and other curricular 
resources; making efforts to improve teachers’ math skills; 
aligning the scope and sequence of instruction in different 


classes; and investing in professional development (PD) to 
support new instructional practices, PL communities, or 
coaching (Desimone et al., 2019; Kane et al., 2016). Studies 
of materials, curriculum, and textbooks have not found sig- 
nificant effects on student outcomes from changing to a new 
textbook in response to standards, or using a curriculum that 
was designed pre- versus post-CCSS-M, although a few par- 
ticular curricula seem to be more effective than others 
(Blazar et al., 2019; Kane et al., 2016). However, there is 
evidence that more PL around the standards is related to 
stronger student outcomes. Kane et al. (2016) found that stu- 
dents scored higher on CCSS-aligned math assessments in 
schools where teachers had more PD around the standards 
and higher percentages of teachers who felt knowledgeable 
about them, controlling for differences in prior-year student 
achievement and teacher effectiveness. 

PD has long been considered a primary lever for enacting 
instructional change (Desimone, 2009; Kennedy, 2016; 
Opfer & Pedder, 2011). Greater investments in PD provide 
legitimacy and consistency to what is being asked of teach- 
ers as well as the opportunity to learn new content and skills 
(Desimone, 2002, 2009; Desimone et al., 2019; Desimone & 
Garet, 2015). As states transitioned to the new standards, 
there were major challenges in providing sufficient high- 
quality PD (Kober et al., 2013; Rentner & Kober, 2014). 
Edgerton (2020) found that many districts lacked the capac- 
ity to provide ongoing support to teachers. Several years 
after standards-adopting states began implementing the 
CCSS-M, only about a third of teachers reported feeling like 
they had high levels of preparation to teach the new stan- 
dards (Kane et al., 2016; Scholastic & Bill & Melinda Gates 
Foundation, 2014). Teachers felt particularly in need of PL 
on differentiating instruction for students with different lev- 
els of prior achievement and helping students achieve the 
practice standards (Hamilton et al., 2016). Variation in the 
use of PL across schools and districts could be a key factor 
for understanding variation in student outcomes with 
CCSS-M implementation. 


Research on Instruction for Heterogeneous Students 
Under the CCSS-M 


While the CCSS guidelines recognize that students with 
disabilities, English learners (ELs), and low-achieving stu- 
dents may need special supports to meet the expectations of 
the new standards (Common Core State Standards Initiative, 
2020a, 2020b), they leave the details of how to provide these 
accommodations up to teachers, schools, and districts. In 
particular, a number of scholars have pointed out the need 
for special attention to the ways students with a track record 
of underperformance in math have experienced the new 
standards (Haager & Vaughn, 2013; Powell et al., 2013; 
Saunders et al., 2013). 


Improvements in Math Instruction and Achievement Around CCSS-M 


Previous research suggests that, absent effective sup- 
ports, students with a history of low achievement in math 
may benefit less than other students or even have worse 
academic outcomes in response to reforms that increase 
rigor in their math classes (Allensworth et al., 2009; 
Clotfelter et al., 2015; Gamoran & Hannigan, 2000; Simzar 
et al., 2016). Students with a history of low achievement 
may become disengaged and stop trying when academic 
rigor increases without simultaneous efforts to help them 
meet the new demands (Bandura, 1986; Bouffard-Bouchard, 
1990; Lent et al., 1984; Shernoff et al., 2003). One of the 
most frequent concerns educators have expressed about the 
CCSS-M is lack of confidence in their ability to help stu- 
dents with low prior achievement attain the skills expected 
by the new standards (Haager & Vaughn, 2013; Hamilton 
et al., 2016; Powell et al., 2013; Saunders et al., 2013). 

At the same time, the standards were adopted to ensure 
that all students have access to a rigorous curriculum and 
instructional practices that promote a deep understanding of 
mathematics. Low-achieving students frequently have been 
grouped in classes that focus heavily on procedures and 
computation (National Council of Teachers of Mathematics, 
2018). Divorced from instruction to support comprehension 
of key mathematical concepts, this type of instruction has 
been shown to make students less motivated to discern the 
reasoning behind the procedures they use (Boaler et al., 
2000; Hiebert & Lefevre, 1986). Over time, students become 
less equipped to undertake rigorous math problems, leading 
to the observed widening of gaps between low and high 
achievers over time (Jordan et al., 2009; Niemi, 1996; 
Richland et al., 2012). For low-achieving students with low 
conceptual understanding of math, instruction to support 
these practice standards may be especially critical. 


How This Study Contributes 


In this study, we examine changes in students’ instruc- 
tional experiences and achievement after the adoption of 
new standards, based on the extensiveness of teachers’ PL 
around the standards across schools in Chicago. We exam- 
ine a wider variety of student outcomes than in existing 
research on CCSS-M, including test scores, grades, rates 
of course passing, and student reports of engaging in stan- 
dards-aligned instructional practices. After examining 
average outcomes of the CCSS-M implementation across 
students, we disaggregate the analyses for students with 
low, average, and high prior-year achievement. The “low- 
achieving” group includes most students with disabilities 
at each grade, as well as students facing other obstacles, 
broadly conceived, that contribute to below-grade level 
math achievement. Arguably, this group contains those 
students whom standards-based reforms aim to help the 
most, but who previous studies suggest will be most likely 
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to struggle with the increased academic demand of the 
new math standards. 

Like the Kane et al. (2016) study, we use variation in par- 
ticipation in PL around the CCSS-M to examine the relation- 
ship between PD and changes in student outcomes. We also 
incorporate a longitudinal component, comparing differ- 
ences in outcomes in poststandards years relative to prestan- 
dards years, similar to Song et al. (2019), Loveless (2014, 
2016), and Bleiberg (2020). As with those studies, and with 
the Kane et al. (2016) study, causal inferences are limited 
because the schools with different levels of implementation 
are not random, just as the states with weaker standards 
pre-CCSS-M were not random in the studies of states. We 
include school fixed effects in the models so as to compare 
schools with themselves in prior years; this provides more 
confidence that changes in outcomes across schools with 
different levels of participation in PL were not simply a 
result of initial differences between schools. 

We are able to do this work using detailed data on stu- 
dents in the middle grades (Grades 6 to 8) from the Chicago 
Public Schools (CPS; n.d.), a large urban district with about 
500 schools that implemented the CCSS-M through a teacher 
leader model that provided PD and resources to encourage 
instructional change around the CCSS-M. The decentralized 
organizational structure of schools in the district resulted in 
significant variation in schools’ participation in the district’s 
PD standards implementation strategy. We leverage this 
variation to examine the relationship between PD on the 
CCSS-M and changes in student outcomes. 


Standards Implementation in Chicago Public Schools: 
Promoting High-Quality Instruction Through Teacher 
Leaders 


After Illinois adopted the CCSS-M in 2010, the CPS 
Department of Mathematics launched a multiyear effort, 
beginning in 2013, to support teachers’ transition to the new 
standards. Their focus was on helping teachers modify their 
instructional practices so that students would develop the 
procedural and conceptual understanding called for by the 
practice standards. As described by one district leader to a 
member of the study team in an interview for an associated 
qualitative study, “this [strategy] was going to be about high- 
quality instruction. Then if the materials changed, the stan- 
dards changed, if anything, the assessment changes, it 
doesn’t matter because this is still good teaching.” The dis- 
trict partnered with local universities to develop and provide 
content for PD sessions, which was informed by Charlotte 
Danielson’s Framework for Teaching (Danielson, 2009) and 
the Teaching for Robust Understanding (TRU) Framework 
(Schoenfeld & Teaching for Robust Understanding Project, 
2016), and it emphasized creating equitable, student- 
centered learning environments. 


A primary feature of their PL program was a teacher 
leader model. Two or three teachers from each school were 
designated teacher leaders and attended Teacher Leader 
Institutes. In schools with higher proportions of low-achiev- 
ing students, teacher leaders and their administrators were 
offered the opportunity to participate in math team learning 
communities where they could develop school-based PL 
plans and share across schools. All teacher leaders were 
expected to develop their own classroom practice to better 
align with the goals of the CCSS-M and support colleagues 
in their schools to make instructional improvements. The 
district offered a number of voluntary standards-focused 
workshops and opportunities for PD and collaboration that 
teachers could attend if they were not designated teacher 
leaders. The district also provided resources to support 
instruction and a list of recommended core curricula through 
a website called the “Knowledge Center.” PD sessions often 
helped teachers use the materials provided by the Knowledge 
Center and the resources it contained to support PL within 
schools. 

CPS schools differed considerably in the degree to which 
their teachers participated in PL around the standards 
(Gwynne & Cowhy, 2017), similar to what has been observed 
in studies of other places (Desimone et al., 2019; Kane et al., 
2016). This variation allows us to analyze the relationship 
between the extensiveness of standards-related PL and 
changes in math instruction and student learning outcomes. 


Method 
Sample and Data 


Our analyses are based on CPS students who were in the 
sixth through eighth grades at any point from 2011-2012 to 
2016-2017. Students are included if they had a math score 
on the state assessment from the previous school year and 
were enrolled in regular CPS schools, which include neigh- 
borhood schools, magnet schools, and selective schools, but 
exclude alternative schools for dropout recovery and special 
education schools. Charter schools are also omitted from the 
analyses presented in the main text because we lacked their 
data on course grades and course passing rates. 


Survey Data. A number of our variables come from surveys 
of teachers and students administered in the spring of each 
year in CPS. Participating in surveys is optional for individ- 
uals. However, schools receive reports on a subset of mea- 
sures that are used for school improvement planning and 
district accountability. Thus, participation is strongly encour- 
aged and time is usually set aside for teachers and students 
to take the surveys during the school day. Many of the 
survey measures show strong evidence of validity in their 
relationships with growth in student achievement over time 


(Klugman et al., 2015; Nomi & Allensworth, 2013). As a 
result of the strong district support of the surveys, response 
rates are high. Teacher response rates in 2015 were 80.7%. 
Student survey response rates ranged from 74% to 83% for 
the surveys administered in the spring of 2011 through 2017. 


Measure of Professional Learning. Our main independent 
variable, other than school year, is the extent to which teach- 
ers in each school participated in the district-wide PL pro- 
gram designed to support the new standards. Teachers’ 
participation in PL is captured with the teacher survey 
administered in the spring of 2015, which was the first year 
(2014—2015) that all teachers in the district were expected to 
use standards-aligned instruction. Several questions asked 
teachers to describe their PD related to the new standards. 
See online Supplemental Appendix A for specific survey 
items. These were combined into a single measure with a 
reliability of 0.76 using a Rasch model (Wright & Masters, 
1982). Since teacher responses on the survey are anonymous 
and cannot be tied to individual students and classrooms, we 
aggregated their responses to the school level to create a 
schoolwide measure of standards-related PD. Using hierar- 
chical linear models, we calculated empirical Bayes esti- 
mates for each school, using a measurement model at Level 
1, with teachers nested within schools at Levels 2 and 3. The 
school-level reliability of the measure was 0.72. 

Using the empirical Bayes estimates from the hierarchi- 
cal linear models, schools were categorized into three equal 
groups, henceforth identified as schools with extensive, 
average, or limited standards-related PL. The categories 
were determined including schools with middle grades and 
high schools so that parallel work on high schools would be 
comparable; this results in uneven numbers of schools in the 
categories for this study. High school teachers participated 
less extensively in PL around the standards than teachers in 
the middle grades. The groups reflect variation in participa- 
tion across schools in PL through the district’s teacher leader 
model, variation in any additional PD teachers experienced 
through other PL opportunities, and variation in school- 
based PD sessions run by teacher leaders. Responses to sur- 
vey items across the three implementation groups are 
provided in online Supplemental Appendix B. 

Schools with more extensive PL had slightly lower 
achievement, served students from neighborhoods with 
higher levels of poverty, and served more Black students 
(see Table 1). This corresponds with district efforts to offer 
additional PD supports for schools with higher proportions 
of low-achieving students. At the same time, the differences 
in mean achievement and student characteristics between 
the three groups are fairly small, relative to the variation 
within the groups. All three groups of schools are heteroge- 
neous along all dimensions of student body composition, 
with considerable overlap across them on all student charac- 
teristics. Moreover, all of the statistical models control for 
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differences in student composition, and the use of school 
fixed effects provides additional confidence that the changes 
we observe in outcomes across schools in each PL category 
were not a result of initial differences between schools. 


Student-Level Variables. District administrative records 
include information about students’ background characteris- 
tics (race/ethnicity, gender, special education status, and 
census block group) for all students enrolled in the district 
each year. To control for students’ socioeconomic status, we 
use variables created from U.S. Census data measuring 
neighborhood poverty (percentage of families under the 
poverty line and percentage of males unemployed) and 
neighborhood social status (median family income and aver- 
age education levels), linked by their residential census 
block group. These allow a more fine-tuned measure of fam- 
ily background than an indicator of free and reduced lunch 
status, since more than 80% of students in CPS qualify. The 
Census measures are standardized across all census block 
groups in the city, not just the blocks in which CPS students 
live. 


Student Reports of Math Practices. For each survey admin- 
istration, students are asked questions specifically about 
practices they do in their math class (Math Instructional 
Practices). These practices correspond with the types of 
activities that are aligned with the CCSS-M Standards for 
Mathematical Practice and are consistent with the practices 
emphasized in district-sponsored PD around the standards. 
The questions were combined into a Rasch measure with a 
reliability of 0.57. See online Supplemental Appendix A for 
the specific survey questions. 


Student Achievement. We use student transcript data for 
information on math course grades and whether students 
passed their math course. Course grades are available by 
semester, and some students take multiple math classes. We 
take the average for each student of their grades in math 
(measured on a 0- to 4-point scale) in each semester class. 
We calculated their math course pass rate based on whether 
they passed (1) or failed (0) each semester of each math 
course they took. 

During the period covered by this study (2011-2017), 
CPS students took a number of different standardized tests. 
The old state test (ISAT [Illinois Standards Achievement 
Test]) was administered through the spring of 2014. The 
PARCC [Partnership for Assessment of Readiness for 
College and Careers], which was developed as an assess- 
ment for the CCSS-M, was administered beginning in the 
spring of 2015. Thus, the change in the state test occurred in 
the same year as standards implementation. Another math 
assessment, the NWEA-MAP (Northwest Evaluation 
Association Measures of Academic Progress), was given to 
all third- through eighth-grade students in every year from 


TABLE 1 


Characteristics of Schools in the Three Professional Learning Groups 


Professional Number 

learning in 2015 of schools Characteristic M SD Min Max 

Limited 110 Prior-year average math score 0.09 0.61 —0.84 2.32 
Neighborhood poverty 0.22 0.56 —1.69 1.43 
Percent White 10% 17% 0% 67% 
Percent Black 41% 40% 0% 100% 
Percent Latino 43% 38% 0% 99% 
Percent special education 14% 7% 0% 34% 
Percent taking algebra 6% 9% 0% 39% 
Professional learning around the standards —1.34 0.57 —3.06 —0.64 

Average 172 Prior-year average math score —0.04 0.46 —0.80 1.63 
Neighborhood poverty 0.31 0.60 =1,25 1.43 
Percent White 9% 18% 0% 73% 
Percent Black 47% 44% 0% 100% 
Percent Latino 40% 39% 0% 100% 
Percent special education 15% 6% 0% 34% 
Percent taking algebra 6% 9% 0% 49% 
Professional learning around the standards —0.16 0.25 —0.64 0.26 

Extensive 186 Prior-year average math score =0.15 0.47 —1.07 2.02 
Neighborhood poverty 0.48 0.54 —1.09 1.52 
Percent White 5% 12% 0% 81% 
Percent Black 58% 44% 0% 100% 
Percent Latino 34% 39% 0% 100% 
Percent special education 15% 6% 0% 38% 
Percent taking algebra 4% 7™% 0% 35% 
Professional learning around the standards 0.94 0.54 0.26 3.02 


Note. Means are calculated based on the student population in the 2014—2015 school year. 


2013 onward. This exam has more precision than the state 
tests because it is computer adaptive. It also correlates with 
the ISAT and PARCC at between 0.86 and 0.90, depending 
on grade and year, disattenuated for measurement error, 
based on our analysis of CPS data. Because the NWEA was 
given in years before and after standards implementation, 
we use NWEA scores as our outcome measure of math 
skills. However, so that we have a prescore measure for the 
year 2013, we use students’ performance on the ISAT or 
PARCC from the prior year (2011 through 2016) as a control 
variable for prior achievement, standardized by grade and 
year. Online Supplemental Appendix C provides an analysis 
of change in PARCC scores over the post-CCSSM years by 
the extensiveness of PL in the school. 

Table 2 provides details on variables for those students 
who have data on all outcomes, which is the sample used for 
the analyses presented in the main text, as well as the full 
data set. We limit the main analysis to students with com- 
plete data so the results are comparable across outcomes; the 
same students are included for each analysis. However, this 
substantially reduces the analytic sample down from the 


population of students in the district. We have test score data 
for almost all students in the district, but students who trans- 
ferred midyear do not have spring test data. We have tran- 
script data for all students except those enrolled at charter 
schools. Survey response rates are about 80%. By including 
only students with data on all four outcomes, the analytic 
sample consists of 213,965 students in 418 schools versus 
325,275 students in 478 schools. The characteristics of the 
reduced and full samples are very similar. We ran models for 
each outcome with the entire population of students who 
have data on the outcome to confirm that our conclusions 
would be the same, as described in online Supplemental 
Appendix C. 

To answer the second research question, we divide the 
data based on students’ prior-year achievement levels. 
Students in the low-achievement group scored at least a half 
standard deviation below the district average for their grade 
level on the prior-year state assessment, those in the high- 
achievement group had scores at least half a standard devia- 
tion above average, and the remainder were in the average 
achievement group. There are several differences other than 


TABLE 2 
Descriptive Statistics on Students in All Years (2013-2017) 


Students Students Students with Students with Students with 
with with low initial average initial high initial 
complete data any data achievement achievement achievement 
Variable Variable 
type name M SD M SD M SD M SD M SD 
Outcomes Instructional practices 0.00 1.00 —0.01 1.01 —0.08 1.09 0.02 1.00 0.05 0.89 
NWEA-MAP score 0.00 1.00 —0.03 1.01 —0.90 0.78 —0.03 0.61 0.93 0.71 
GPA 2.58 1.05 2.55 1.06 1.93 0.96 2.51 0.93 3.32 0.79 
Math pass rate 0.96 0.21 0.95 0.21 0.90 0.30 0.97 0.18 0.99 0.07 
Student Concentrated poverty 0.19 0.81 0.23 0.80 0.36 0.76 0.22 0.78 —0.02 0.84 
covariates Social status —0.45 0.90 —0.46 0.88 —0.58 0.85 —0.51 0.87 —0.26 0.94 
Prior math score 0.05 1.00 0.01 1.00 —1.05 0.42 -0.01 0.28 1.22 0.62 
Grade 7 33% 33% 32% 34% 33% 
Grade 8 33% 33% 32% 34% 33% 
Black 33% 37% 44% 34% 22% 
White 10% 9% 4% 7™% 19% 
Asian 4% 4% 1% 3% 9% 
Latinx 51% 50% 50% 55% 48% 
Other race or ethnicity 1% 1% 1% 1% 2% 
Male 49% 50% 52% 47% 48% 
Identified disability 12% 13% 29% 7% 2% 
Enrolled in algebra 7% 6% 1% 5% 17% 
Other types of math 0% 0% 0% 0% 1% 
Multiple types of math 3% 3% 1% 3% 6% 
School Limited PL in 2015 22% 24% 20% 21% 25% 
level Average PL in 2015 39% 38% 38% 39% 41% 
Extensive PL in 2015 39% 38% 42% 40% 34% 
Total n 312,364 442,013 95,648 119,805 96,911 


Note. Low initial achievement is less than 0.5 SDs below the district average on the prior-year state assessment in math for student grade level; high initial 
achievement is at least 0.5 SD above; average initial achievement is within a half standard deviation. Instructional practices and NWEA scores are standard- 
ized around the 2013 mean and standard deviation. NWEA-MAP = Northwest Evaluation Association Measures of Academic Progress; GPA = grade point 


average; PL = professional learning. 


their prior-year math scores (see Table 2). On average, stu- 
dents in the public schools in Chicago tend to live in areas of 
the city with higher poverty rates than the city as a whole. 
However, students with high prior-year achievement come 
from census blocks with poverty levels that are at the city 
average. These students may be more representative of state 
or national population averages in terms of students’ eco- 
nomic status than the other groups. Students in the group 
with low prior-year achievement are much more likely to 
have identified disabilities (29% vs. 2% in the high-achieve- 
ment group). 


Analytic Methods 


We take a difference-in-differences (DiD) approach, com- 
paring changes in student outcomes in postimplementation 
years to preimplementation years, in schools with extensive 
and moderate implementation of PL around the standards to 
those with limited PL. All schools were expected to 


implement the standards beginning in the 2014-2015 school 
year. This is also the year in which we measured teachers’ 
participation in PL around the standards. We refer to 2015 as 
the year when the standards and the PL around the standards 
were implemented, with 2011 to 2014 as preimplementation 
years and 2015 to 2017 as postimplementation years. 

The first difference that we model is based on school 
year. We compare outcomes in the spring of 2015, 2016, and 
2017 (postimplementation years) with outcomes in the 
spring of 2014 to discern whether student outcomes were 
higher in postimplementation years. Each year is modeled as 
its own dummy variable, with 2014 as the excluded group, 
so that we can observe whether any changes were short- 
lived, sustained, or grew over time. If the CCSS-M brought 
changes in instruction and student outcomes, we should see 
that outcomes are different in 2015 and later years, relative 
to 2014. We include a series of student covariates so that we 
compare the outcomes of students with the same back- 
grounds over time. 


Allensworth et al. 


Because other initiatives were occurring in the district at 
the same time that it adopted the CCSS-M, we cannot fully 
attribute any of the changes observed in student outcomes in 
postimplementation years to standards implementation. 
Therefore, we focus on teacher participation in PL around 
the standards in the 2014-2015 school year as one aspect of 
standards implementation, discerning whether schools with 
extensive or average participation in PL saw greater changes 
in student outcomes than schools with limited PL. Because 
teacher participation in PL was minimal in schools with lim- 
ited PL, the changes we observe postimplementation for 
schools with limited PL mostly reflect other changes hap- 
pening in the district. Thus, the differences estimated for 
these schools minus the differences observed in the other 
schools with different levels of PL reflect the effects of par- 
ticipation in PL around the standards. This approach pro- 
vides more confidence that observed changes were due to a 
component of standards implementation rather than other 
changes occurring in the district. 

One concern with this approach is that the schools that 
participate more fully in PL around the standards are differ- 
ent in some unmeasured ways from schools with limited PL, 
and that these differences might also be related to student 
outcomes. To mitigate this concern, we include school fixed 
effects that capture stable characteristics of schools. Thus, we 
are comparing school outcomes in postimplementation years 
with outcomes in the same schools preimplementation— 
comparing schools with themselves. We include dummy 
variables for each school year with 2014 as the excluded 
group, alone and interacted with PL. If the groups are com- 
parable, with the covariates and school fixed effects, we 
should see no significant difference in the years before 
implementation, relative to 2014 (the year before implemen- 
tation). We should see differences in 2015 and later years, 
relative to 2014, if PL in 2015 was related to improvements 
in student achievement. The model is as follows: 


6 18 
Yist =) Biyear, + ) yeyear, * I, + Os + QZist + sist 
c=7 


t=1 
The outcome Y for student i in school s in time t, where 


B, to B, capture the differences in the outcome in each 
year compared with 2014 through a series of dummy 
variables for time (D2011—D2017, excluding 2014); 

I, are dummy variables for two of the implementation 
groups—schools with average PL and schools with 
extensive PL; 

y, are the DiD coefficients for schools with different 
levels on the intervention, / (either extensive or aver- 
age implementation). Among schools with limited 
PL (the excluded group), the difference in the out- 
come from 2014 to 2015 is the main effect for D2015, 
B,, while for schools with average PL, the difference 


is 8, plus the coefficient on the interaction of D2015 
* ImpAve, 710: 

a, are school fixed effects; 

Z;, are student-specific covariates; and 


standard errors were clustered by school. 


We also ran a model that used a continuous variable of PL, 
rather than categories of limited, average, and extensive PL 
(see online Supplemental Tables C4 and CS in Appendix C). 
We highlight the categorical models since they are used to 
make a figure and table that are easier to interpret. 

We then run the same models separately for each of the 
three subgroups defined by students’ achievement in the 
prior year to discern whether there were differences in out- 
comes for students with different prior-year achievement 
levels. By running the models separately, the covariates are 
specific to each of the subgroups, and the estimates are not 
influenced by outcomes for students in the other achieve- 
ment groups. 


Limitations 


There are a number of caveats to this study. First, the 
schools with extensive PL were not a random group. 
Administrators could elect to have their schools participate 
in standards-related PL, and teachers could elect to partici- 
pate. Those who participated more fully may have been 
more receptive to PL. Online Supplemental Appendix B pro- 
vides more information on the three implementation groups. 
Each group contained a heterogeneous mix of schools, but 
schools with more extensive participation had somewhat 
lower achievement, on average. Their teachers might have 
felt a greater press for change. Requiring extensive PL in 
schools with less buy-in might produce different results than 
shown here. 

Second, the magnitude of change due to PL may be an 
underestimate of the total effects because we only examine 
differences relative to other schools—all of whom received 
some degree of PL around the standards. As shown in online 
Supplemental Appendix B, not all teachers in schools with 
extensive PL participated in standards-related PD, while a 
small percentage of teachers in schools with limited PL par- 
ticipated regularly in PL. The relationships we measure are 
further diminished by measurement error that exists in our 
school-level measure of teachers’ participation in PL. 

Third, these changes are occurring within a context of 
other policy changes in the district, including a new teacher 
evaluation system, and restrictions on the use of suspensions 
coupled with more restorative discipline strategies. While 
we leverage differences in participation in standards-based 
PL to tease out the influence of standards-based support, it 
could be that standards-based PL was effective only because 
these other policy elements were also in place, or were in 
some way due to these other district policies. Fourth, these 


methods isolate one particular element of implementation 
(PL) and do not identify the total influence of the stan- 
dards. There is not a way to tease out the results of the 
complete district implementation plan from other district- 
wide changes. These are also conservative estimates of the 
influence of PL around the standards, as we measure par- 
ticipation in PD in the first year of district-wide standards 
implementation, while student outcomes are measured 
through 3 years of implementation. We do this so the treat- 
ment variable (teacher reports of participation in PD in the 
2014-2015 school year) occurs prior to the time any outcome 
variables would be observed (spring 2014-2015 through 
2016-2017). 


Results 


Coefficients from the first set of models are shown in 
Table 3. The year fixed effects in the first several rows show 
how students’ outcomes changed in each year, relative to 
2014, among schools with limited PL around the standards. 
There were improvements in most outcomes over time, but 
we cannot determine whether they were due to the standards, 
or to other district initiatives. The next two sets of coeffi- 
cients include interaction terms between year and whether 
schools engaged in average or extensive levels of PL around 
the standards. These coefficients show the extent to which 
outcomes changed more in schools with average or exten- 
sive PL around the standards than in schools with limited 
PL. These coefficients should be added to the main year 
fixed effects to determine the overall change in the outcome 
for schools with average or extensive PL around the stan- 
dards, relative to 2014. To make this easier to see, Figure | 
graphs these changes over time. The 2011 to 2014 values in 
the figures show how much the schools started off in differ- 
ent places on the outcomes, but the school fixed effects con- 
trol for any differences when estimating the DiD coefficients 
in the models. 

In general, schools with extensive PL around the stan- 
dards in 2015 showed significantly more growth in student 
outcomes than schools with limited PL in 2015 and later 
years, while schools with average PL showed few differ- 
ences from schools with limited PL. Neither group showed 
significantly different growth compared with schools with 
limited PL in 2011 to 2014, the years before implementa- 
tion. Starting in 2015, students in schools with extensive 
PL reported significantly greater improvement in Math 
Instructional Practices than students in schools with lim- 
ited PL around the standards (0.080 SD). The differences 
relative to schools with limited PL were sustained over 
time. Students in schools with average levels of PL also 
reported significantly larger improvements in instructional 
practice than schools with limited PL, but these differences 
were only significant in 2016. 

Student academic outcomes also improved more in 
schools with extensive standards-related PL than those with 


Improvements in Math Instruction and Achievement Around CCSS-M 


limited PL. Math course grades improved by 0.087 GPA 
(grade point average) points more in schools with extensive 
PL than in schools with limited PL in 2015, with the differ- 
ence growing to 0.130 GPA points by 2017. Schools with 
average standards-related PL did not show significantly 
larger improvements in GPA than schools with limited PL. 
Math course pass rates improved more in schools with both 
average and extensive PL compared with schools with lim- 
ited PL, improving by | to 2 percentage points more in 2015, 
2016, and 2017. NWEA scores grew more in schools with 
extensive PL than in schools with limited PL in 2016 and 
2017 (0.061 SD in 2016 and 0.069 SD in 2017), but scores 
were not significantly higher in schools with average PL 
than in schools with limited PL. The differences in NWEA 
scores in schools with extensive PL are similar to impacts on 
PARCC scores found by Kane et al. (2016), who found that 
an additional 2 days of PL each year improved PARCC 
scores by 0.045 SD. 


Differences by Students’ Prior Achievement 


Looking at all students together obscures differences 
based on students’ initial achievement levels. Schools with 
extensive PL showed significantly larger improvements in 
achievement than schools with limited PL among their low- 
and average-achieving students, but there were few signifi- 
cant differences in the achievement of their high-achieving 
students (see Table 4). For students with low initial achieve- 
ment, math GPAs improved by 0.161 points more in 2017, 
and pass rates improved by 3.1 percentage points more, if 
they were in schools with extensive PL than in schools with 
limited PL. Low-achieving students’ NWEA scores 
improved by 0.078 SD more in schools with extensive PL 
than in schools with limited PL in 2016, relative to scores in 
2014. Low-achieving students also showed more improve- 
ments in grades and pass rates in schools with average PL 
around the standards than in schools with limited PL. At 
average-PL schools, low-achieving students’ GPAs improved 
by more than 0.10 points in 2015 and 2017, and pass rates 
improved by about 3 additional percentage points in all post- 
implementation years, relative to grades and pass rates in 
2014. 

Average-achieving students also showed significantly 
greater improvements in test scores, math course grades, and 
pass rates in schools with extensive PL than in schools with 
limited PL, with test scores improving 0.093 SD more, 
grades by 0.151 points more, and pass rates by 1.4 percent- 
age points more. Average-achieving students did not show 
significantly more improvements if in schools with average 
levels of PL, with the exception of showing higher pass rates 
in 2016 by 1.3 percentage points. 

Table 5 shows the results as averages, rather than coeffi- 
cients that need to be added together, and shows how the 
groups were different in the base year. The values for 2014 
are the simple means for each subgroup, based on students’ 
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Student outcomes by the extensiveness of professional learning. 


Note. Values for 2014 come from regression models that control for student race, ethnicity, gender, neighborhood poverty level, neighborhood social status, 
the type(s) of math class in which the student was enrolled, and math score on the state assessment in the prior spring. Changes relative to 2014 are based 
on coefficients from models with school and year fixed effects and the same covariates. Schools with average professional learning (PL) in 2015 showed 
significantly more growth than schools with limited PL in math practices in 2016 and pass rates in 2015, 2016, and 2017, relative to 2014. Schools with 
extensive PL in 2015 showed significantly more growth than schools with limited PL in math practices in 2015 and 2017, NWEA-MAP (Northwest Evalua- 
tion Association Measures of Academic Progress) scores in 2016 and 2017, GPAs (grade point averages) and pass rates in all years. 


prior-year achievement, year, and school PL around the stan- 
dards. The values for other years are calculated by adding 
the coefficients from Table 4 onto the 2014 subgroup means, 
with coefficients from interaction terms added to the year 
fixed effects for students in schools with average or exten- 
sive PL. 

Prior to standards implementation, students with high 
prior-year achievement reported more frequently engaging 
in standards-aligned instructional practices than low-achiev- 
ing students (by 0.23—-0.25 SD in 2011 and 0.11-0.14 SD in 
2014; see Table 5). In the postimplementation years, low- 
achieving students showed increases in standards-aligned 
practices of 0.15 SD in schools with limited PL and 0.24 SD 
in schools with extensive PL. Students with low achieve- 
ment still reported less engagement in standards-aligned 
practices than students with high prior-year achievement in 
the same schools, but the differences were smaller, and low- 
achieving students in schools with extensive PL engaged in 
standards-aligned practices at higher rates than high-achiev- 
ing students in schools with limited PL after standards 
implementation. 


Overall, achievement was still lower among students who 
had lower prior achievement than other students, but in 
schools with extensive PL, the differences in GPA and pass 
rates between students with low and high test scores were 
narrower in postpolicy years than in prepolicy years. The 
difference in GPAs was 0.11 points smaller, and the differ- 
ence in pass rates was 4 percentage points smaller. 
Differences in NWEA scores remained similar. 


Alternative Specifications 


Results from alternative models are presented in online 
Supplemental Appendix C; the conclusions are similar to 
those in the main text. Coefficients from models that use all 
cases with data on each outcome are included in online 
Supplemental Tables C1 to C3; the DiD coefficients in these 
models are larger and more consistently significant for pre- 
dictions of math practices and NWEA scores among low- 
and average-achieving students. Coefficients from models 
that use a continuous measure of PL are shown in online 
Supplemental Tables C4 and C5; the patterns of which DiD 
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TABLE 5 


Average Outcomes by Prior-Year Student Achievement, School Professional Learning Around the Standards and Year, Based on Model 


Estimates 


Students with Schools with 2011 (Pre) —0.30 iS 87 
low prior-year limited PL 2012 (Pre) —0.21 1.90 90 
achievement 2013 (Pre) -0.17 -1.04 1.95 91 

2014 (Pre) -0.13 0.96 1.96 93 

2015 —0.05 —0.83 1.99 93 

2016 —0.04 —0.80 2.09 95 

2017 0.02 —0.78 2.08 95 

Schools with 2011 (Pre) -0.27 1.78 87 
average PL 2012 (Pre) -0.19 1.83 87 
2013 (Pre) 0.15 1.06 1.87 88 

2014 (Pre) =on2) ~0.94 1.86 89 

2015 —0.02 —0.85 2.02 93 

2016 0.10 —0.78 2.08 95 

2017 0.09 —0.74 2.08 94 

Schools with 2011 (Pre) —0.24 1.74 84 
extensive PL 2012 (Pre) —0.20 1.84 87 
2013 (Pre) —0.08 el OF 1.84 87 

2014 (Pre) 0.07 -0.94 1.89 90 

2015 0.14 —0.81 2.06 93 

2016 0.13 —0.70 2.18 96 

2017 0.17 —0.67 2.16 95 

Students with Schools with 2011 (Pre) —0.14 2.34 95 
average limited PL 2012 (Pre) -0.10 DS 97 
prior-year 2013 (Pre) 0.00 0.11 2.54 97 
achievement 2014 (Pre) 0.04 0.05 2.56 98 

2015 0.05 —0.03 2.56 97 

2016 0.05 0.00 2.63 98 

2017 0.05 —0.03 2.59 98 

Schools with 2011 (Pre) =O, 118) 2.41 95 
average PL 2012 (Pre) —0.06 2.42 95 
2013 (Pre) 0.01 -0.11 2.50 97 

2014 (Pre) 0.02 ~0.04 2.50 97 

2015 0.08 0.00 2.51 97 

2016 0.14 0.03 2.56 98 

2017 0.13 0.00 2.60 98 

Schools with 2011 (Pre) els) 232 94 
extensive PL 2012 (Pre) —0.08 2.42 95 
2013 (Pre) 0.03 —0.14 2.45 96 

2014 (Pre) 0.03 —0.05 2.46 Oi/ 

2015 0.17 0.00 2:53 97 

2016 0.18 0.06 2.66 98 

2017 0.23 0.06 2.64 98 

Students with Schools with 2011 (Pre) —0.05 3h) OY) 

high prior-year limited PL 2012 (Pre) —0.02 327 ge) 
hi tt 
ee 2013 (Pre) 0.09 0.85 3.33 100 
2014 (Pre) ~0.01 0.91 3.38 100 
(continued) 
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TABLE 5 (CONTINUED) 


Student Math practices (in © NWEA score (in GPA Pass math (in 
achievement School PL Year SD units) SD units) (in GPA points) percentage points) 
2015 0.06 1.00 3.37 100 
2016 0.10 1.06 3.45 100 
2017 0.04 12 3.46 100 
Schools with 2011 (Pre) —0.04 3)228) 99 
average PL 2012 (Pre) —0.04 3.26 99 
2013 (Pre) 0.04 0.77 3.29 99 
2014 (Pre) ~0.02 0.86 3.31 99 
2015 0.06 0.96 3.31 100 
2016 0.16 1.04 3.37 100 
2017 0.12 1.09 3.40 100 
Schools with 2011 (Pre) —0.01 3.18 oe) 
extensive PL 2012 (Pre) 0.01 3.24 OY) 
2013 (Pre) 0.09 0.73 3.28 99 
2014 (Pre) 0.07 0.79 Cw] 99 
2015 0.20 0.90 3.30 100 
2016 0.17 0.99 3.40 100 
2017 0.20 1.06 3.43 100 


Note. Values for 2014 are simple averages for each subgroup. Values for other years use coefficients from models controlling for school and year fixed 
effects, student race, ethnicity, gender, neighborhood poverty level, neighborhood social status, the type(s) of math class, and prior-year math score, added 
on to the 2014 values. NWEA = Northwest Evaluation Association; GPA = grade point average; CCSSM = Common Core State Standards in Mathematics; 


PL = professional learning. 


coefficients are significant are similar to those of schools 
with extensive PL. Online Supplemental Table C6 provides 
an analysis of PARCC scores; these were significantly 
higher in 2016 for schools with average PL, and in both 
2016 and 2017 in schools with extensive PL. Table C8 
includes teacher fixed effects; this model has the potential to 
confound school PL and teacher effects, but there are signifi- 
cant relationships on a number of the DiD coefficients, and 
the size of the coefficients are similar to models without 
teacher fixed effects. 


Discussion 


While the CCSS identify a centralized set of knowledge 
and skills that students should master, strategies for imple- 
menting the new standards are intentionally left to districts 
and schools. This design has led to considerable variation 
in implementation, prompting calls for research on how 
district and school supports affect classroom practice and 
academic outcomes for various subgroups of students 
(Polikoff, 2017). In Chicago, the district developed a PL 
program that emphasized changing instructional practices 
in math through a teacher leader model. Evidence from 
this study suggests that the strategy seems to have paid off, 
increasing student engagement in instructional practices 
consistent with the new standards, reducing differences in 


instructional experiences among students with different 
prior achievement levels, and showing improvements in 
test scores, grades, and pass rates in math classes. 
Achievement improved district-wide during this period 
for students of all achievement levels, but the improve- 
ments were greatest in schools with extensive participa- 
tion in PL, particularly among students with low prior-year 
achievement. The patterns observed in Chicago are in line 
with the goals of the CCSS-M to improve instructional 
equity, closing achievement gaps in math course pass rates 
and grades. 


The Benefits to Achievement Were Observed Most 
Consistently for Students With Low Achievement and Those 
at Risk of Failing Their Math Class: Why? 


Study after study have shown that teachers worry about 
being able to teach to the new standards for low-achieving 
students, and they find it difficult to do so (Haager & 
Vaughn, 2013; Hamilton et al., 2016; Powell et al., 2013; 
Saunders et al., 2013). However, even if they find it diffi- 
cult, the benefits may still be large for low-achieving stu- 
dents who were least likely to engage in practices that 
promote deep engagement and conceptual understanding 
prior to the standards. A prior study on double-dose algebra 
in Chicago also found improvements in math scores for 


1) 


Allensworth et al. 


students with below-average math skills. In that case, as in 
the case of CCSS-M implementation, teachers received PD 
for how to do more student-centered instructional practices 
that support conceptual understanding, and the extra time 
gave them the flexibility to take the chance on teaching in 
new ways (Nomi & Allensworth, 2013). Changes in instruc- 
tional practices may also have been less critical for the 
achievement of students who already were successful in 
math. 


Chicago Provides an Example of Successful 
Implementation: Why Is It Different From Studies of 
CCSS-M Adopting States? 


This analysis provides evidence of positive outcomes 
associated with CCSS-M implementation in one city. 
Because most studies analyze states as a whole, we don’t 
know which other districts had successful implementation. 
There are a number of reasons why we might see positive 
results in Chicago when statewide results are null, or show 
improvements mostly for high-SES students—and these 
have implications for future efforts to improve achievement 
and equity in math. 

First, it could be that CPS emphasized what ultimately 
matters most for student engagement and learning, particu- 
larly for low-achieving students. The district’s support 
for new instructional practices meant that teachers were 
prepared to implement not just the content standards but 
also the practice standards. The Teaching for Robust 
Understanding and Danielson frameworks integrated 
into the PD around the standards further supported the 
emphasis on instructional practices, and the availability of 
instruction-ready resources for promoting student discussion 
and problem-solving around multiple solutions made it eas- 
ier for teachers to try new techniques. The prior studies that 
found no positive effects, or positive effects of standards 
only on economically advantaged students (Bleiberg, 2020; 
Song et al., 2019), used change in the content standards as 
the independent variable; they could not speak to how much 
instructional practices changed in those states. It is possible 
that economically advantaged districts had more resources 
to invest in PD. It is also possible that raising content stan- 
dards without improving the quality of instruction does not 
benefit low-achieving students; studies of prior policies 
intended to raise rigor in math classes have found null or 
negative effects on low-achieving students (Allensworth 
et al., 2009; Clotfelter et al., 2015; Gamoran & Hannigan, 
2000; Simzar et al., 2016). Furthermore, the Polikoff and 
Porter (2014) study that found no relationship between 
teachers’ instruction and student gains focused primarily on 
the content standards. 

Second, the Chicago context is not representative of the 
state or nation as a whole; it is an 80% low-income district, 
where many students begin school with low achievement 
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relative to national averages. Significant achievement gains 
were only observed for students with low or average achieve- 
ment relative to other students in Chicago. When results are 
averaged across all students, the benefits for students with 
low achievement may be obscured. Typical districts in the 
state and nation may be more similar to students who are 
high achieving in Chicago. 

The teacher model was also run in a way that is quite dif- 
ferent from PL approaches in other places. This might have 
influenced the success of schools with extensive PL. Rather 
than training teachers to implement curricula in a prescribed 
way, the district encouraged teachers to “try out” practices 
and share what worked with others. This approach—with 
teachers supporting each other around instructional change 
with access to expert knowledge and resources—requires 
experimenting and learning from failure. That can be frustrat- 
ing for teachers and involves considerable uncertainty. But it 
is consistent with learning science and innovation research 
(Cobb & Jackson, 2011; Coburn et al., 2013; Coburn & 
Russell, 2008). It also mirrors the goals of the CCSS-M for 
students, where students are presented problems and given 
support to find different solutions themselves. 

Prior studies also focused solely on test scores as out- 
comes. Improvements in test scores were less clear than 
improvements in grades or pass rates—test scores improved 
gradually over time while grades and pass rates improved in 
the implementation year and then continued to grow. If we 
had only looked at test scores, we would have added another 
study to the literature that shows ambiguous results. The 
standards themselves may not clearly translate into gains on 
standardized tests—or they may require considerable sup- 
port for teachers to make instructional changes that have a 
positive impact and those changes may take time to be seen 
in test scores. 


The Teacher Leader Model Was Successful for Some 
Schools but Not All Schools 


The model seems to have been effective for schools that 
participated in standards-aligned PL most extensively. In 
these schools, improvements began in 2015 and continued to 
accumulate through 2017. This continued improvement may 
reflect continued participation in PL, which was offered by 
the district through 2017. It may also reflect the design of the 
teacher leader model, wherein multiple teacher leaders from 
each school worked together to improve instruction. This 
collaborative design could have improved internal capacity 
for changes that were sustainable over time. 

At the same time, it is important not to lose sight of the 
wide variation in participation in PL across schools. Many 
schools participated in PL to only a limited degree, and the 
achievement gains for schools with average levels of PL 
were largely null relative to schools with limited PL, except 
among low-achieving students. Furthermore, this analysis 


only shows what happened in schools serving the middle 
grades. Initially, we planned to include high schools in our 
analysis. However, only 8% of high schools (11 schools) 
engaged in extensive PL. The estimates of policy effects 
among high schools were not stable with different model 
specifications. This variation in schools’ implementation 
suggests that the teacher leader model was not entirely suf- 
ficient for meeting the district’s aim of reaching all its 
teachers. 
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Appendix A 
Student and Teacher Survey Items 
Participation in CCSS-M-Related Professional Learning (2015 Teacher Survey): 


How often did you receive formal training or professional development on CCSS so far this 
year? 
(1-Never, 2-Once this year, 3-Once or twice a semester, 4-Monthly, 5-Weekly) 


Which of the following topics have been addressed in your CCSS-M training and 
professional development? Check all that apply. 
© Common Core Standards in Mathematics 
© Curriculum materials and resources to teach the Common Core standards 
© Teaching common standards to specific student groups (for example, students 
with disabilities or English Language Learners) 
o Adapting classroom assessments to the Common Core standards 
New standardized assessments aligned with CCSS 
© Research on best practices for implementation of the Common Core standards 


1e) 


Math Instructional Practices (2011-2017 Student Survey) 


In your MATH class this year, how often do you do the following: 
(1-Never, 2-Rarely,3-Sometimes, 4-About half the time the class meets, 5-About every time the class 
meets) 
e Write afew sentences to explain how you solved a math problem. 
Explain how you solved a problem to the class. 
Write a math problem for other students to solve. 
Discuss possible solutions to problems with other students. 
Apply math to situations in life outside of school. 
Solve a problem with multiple steps that takes more than 20 minutes. 
Write a few sentences to explain how you solved a math problem. 
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Appendix B 


Further Information on Schools in the Professional Learning Categories 


At schools with extensive professional learning around the standards, about 35 percent of 
teachers reported engaging in formal PL around the CCSS at least monthly, while only 25 
percent reported engaging in PL once a year or less (see Figure B1). In contrast, at schools with 
limited PL, less than 10 percent of teachers engaged in PL around the standards at least once a 
month, while almost 60 percent did so once a year or less. About 70 percent of teachers at 
schools with extensive PL around the standards reported addressing topics that included research 
on best practices for implementation of the CCSS, new assessments aligned with CCSS, adapting 
classroom assessments to the CCSS, and curriculum materials and resources to teach the CCSS. 
At schools with limited PL, fewer than half of teachers said those topics were addressed in their 


professional learning. 


Figure B1. Frequency of Professional Development around the CCSS-M by School the Overall 
Extensiveness of Professional Learning in Teachers' Schools 
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Figure B2. Teacher Reports of CCSS Topics in their Professional Learning by Extensiveness of 
Professional Learning in their School 
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Appendix C 


Results from Models with Different Specifications 


Replication with Full Data 

To ensure our conclusions were not affected by limiting the analytic sample to students 
with data on all outcomes we also ran the models inclusive of all students who had data on any 
outcome. For the analysis of the math practices survey measure, using the full data allows 
students at charter schools to be included in the analysis. For the analysis of NWEA scores, using 
the full data allows students at charter schools and students who did not take the spring survey to 
be included. For the analysis of grades and pass rates, using the full data allows students who left 
the district before the spring test or the spring survey to be included--analyzing their fall 
semester grades. The results are shown in Tables C.1-C.3 below. 

The patterns are similar, with significant coefficients for all of the outcomes in post- 
implementation years among low- and average-achieving students in schools with extensive 
professional learning, and mostly insignificant coefficients among high achieving students. 
However, the coefficients are larger and more consistently significant for the measures of math 
practices and NWEA scores among students with low and average prior-year achievement in the 
models with all cases than in the models with the reduced sample. As observed with the reduced 
sample, there were fewer and less consistent significant differences between schools with 


average PL and schools with limited PL than between those with extensive PL and limited PL. 
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Table C.1. Results for Low Achieving Students with Full Data 


Year Fixed 
Effects 
(Relative to 
2013) 


Coefficient 
Intercept 
2011 (Pre-CCSSM) 
2012 (Pre-CCSSM) 
2013 (Pre-CCSSM) 
2015 (Post) 
2016 (Post) 
2017 (Post 


201 1xAveragePL (Pre) 


Math Practices 
Estimate $.E. 
-0.179 0.039 *** 
-0.099 0.034 * 
-0.048 0.037 
0.073 0.036 * 
0.059 0.038 
0.127 0.044 * 
0.036 0.048 


NWEA Score 
Estimate S.E. 
-0.210 0.020 *** 


-0.096 0.023 *** 
0.129 0.019 *** 
0.123 0.024 *** 
0.105 0.030 *** 


GPA 
Estimate $S.E. 
2.268 0.025 *** 
-0.223 0.052 *** 
-0.067 0.052 
-0.010 0.032 
0.038 0.032 
0.125 0.034 *** 
0.152 0.037 *** 
0.127 0.062 * 


Pass Math 


Estimate 


-0.065 
-0.020 
-0.015 
0.005 
0.025 


SS. 
0.007 *** 
0.016 *** 
0.013 
0.008 . 
0.011 
0.009 ** 
0.010 ** 
0.019. 


201 2xAveragePL (Pre) 0.035 0.043 0.010 0.063 -0.002 0.017 
Schools with} 2013xAveragePL (Pre) 0.042 0.047 -0.026 0.032 0.000 0.044 0.000 0.012 
Average PL 2015xAveragePL 0.039 0.044 -0.011 0.027 0.097 0.043 * 0.024 0.014. 
2016xAveragePL 0.157 0.049 ** 0.036 0.032 0.078 0.048 0.023 0.013. 
2017xAveragePL 0.075 0.057 0.044 0.039 0.077 0.050 0.022 0.014 
2011xHighPL (Pre) 0.025 0.048 0.069 0.062 0.012 0.020 
2012xHighPL (Pre) -0.018 0.043 0.018 0.061 -0.008 0.017 
Schools with} |= 2013xHighPL (Pre) 0.045 0.045 -0.046 0.030 -0.038 0.044 -0.013 0.012 
Extensive PL 2015xHighPL 0.148 0.054 ** -0.003 0.026 0.126 0.043 ** 0.025 0.013. 
2016xHighPL 0.152 0.046 * 0.094 0.033 ** 0.171 0.044 *** 0.039 0.013 ** 
2017xHighPL 0.114 0.056 * 0.109 0.040 ** 0.136 0.049 ** 0.029 0.013 * 
Covariates Grade 7 -0.047 0.010 *** 0.238 0.010 *** 0.007 0.016 0.004 0.005 
Grade 8 -0.040 0.011 *** 0.533 0.010 *** 0.222 0.015 *** 0.043 0.004 *** 
Algebra 0.052 0.063 0.250 0.056 ***| -0.062 0.067 -0.029 0.010 ** 
Other math 0.808 0.052 *** 0.915 0.031 *** 3.753 0.042 *** 0.957 0.012 *** 
Multiple math -0.012 0.053 0.071 0.059 0.225 0.106 * 0.002 0.012 
White -0.060 0.026 * 0.065 0.018 *** 0.231 0.019 *** 0.031 0.006 *** 
Asian 0.003 0.033 0.136 0.025 *** 0.365 0.032 *** 0.048 0.007 *** 
Hispanic -0.123 0.014 *** 0.038 0.010 *** 0.105 0.014 *** 0.018 0.004 *** 
Other -0.069 0.035 * 0.093 0.025 *** 0.126 0.032 *** 0.012 0.010 
Male -0.044 0.008 *** 0.042 0.005***| -0.266 0.009***) -0.053 0.003 *** 
Poverty 0.005 0.006 -0.012 0.004 ** -0.010 0.005. 0.000 0.002 
Social Status -0.008 0.007 0.002 0.005 -0.002 0.006 0.000 0.002 
Prior math score 0.128 0.008 *** 0.701 0.011 *** 0.176 0.012 *** 0.032 0.003 *** 


Note: .<.10, *<.05, **<.01, ***<.001. Low achievement is defined as at least 0.5 s.d. below the district mean in the 
prior year on the state assessment. Coefficients come from models that also include school fixed-effects. Standard 
errors were clustered by school. Math practices and NWEA scores were standardized using means and standard 
deviations from 2014. Teacher professional learning (PL) around the CCSS-M was measured in the 2014-15 school 
year, which was the same year all schools were to begin implementing the standards. 
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Table C.2. Results for Average Achieving Students with Full Data 


Year Fixed 
Effects 
(Relative to 
2013) 


Coefficient 
Intercept 


2011 (Pre-CCSSM) 
2012 (Pre-CCSSM) 
2013 (PreCCSSM) 


2015 (Post) 
2016 (Post) 
2017 (Post 


201 1xAveragePL (Pre) 


Math Practices 
Estimate $.E. 

-0.109 0.037 * 
-0.074 0.033 * 
0.030 0.030 

0.064 0.031 * 
0.062 0.033 . 
0.067 0.037 . 
0.012 0.048 


Estimate _$S.E. 
-0.189 0.011 ** 
-0.079 0.019 ** 


NWEA Score 


0.012 0.014 
0.021 0.020 
-0.016 0.026 


GPA 
Estimate $S.E. 
2.666 0.020 *** 
-0.246 0.062 *** 
-0.057 0.042 
-0.030 0.027 
-0.021 0.033 
0.051 0.036 
0.025 0.039 
0.153 0.071 


Estimate 


Pass Math 


-0.036 
-0.004 
-0.010 
-0.006 

0.003 


S.E. 
0.002 *** 
0.009 *** 
0.005 
0.003 ** 
0.004 . 
0.004 
0.005 
0.011 


201 2xAveragePL (Pre) 0.039 0.042 -0.028 0.053 -0.017 0.008 * 
Schools with} 2013xAveragePL (Pre) 0.020 0.039 0.010 0.027 0.028 0.036 0.005 0.005 
Average PL 2015xAveragePL 0.042 0.039 0.043 0.019 * 0.032 0.046 0.004 0.005 
2016xAveragePL 0.108 0.043 * 0.059 0.026 * 0.026 0.047 0.009 0.005. 
2017xAveragePL 0.100 0.047 * 0.041 0.032 0.073 0.051 0.009 0.006 
2011xHighPL (Pre) -0.056 0.045 0.101 0.070 0.009 0.011 
2012xHighPL (Pre) -0.024 0.042 0.003 0.051 -0.017 0.007 * 
Schools with} |= 2013xHighPL (Pre) -0.027 0.036 -0.018 0.026 0.013 0.039 0.002 0.005 
Extensive PL 2015xHighPL 0.088 0.044 * 0.043 0.021 * 0.079 0.042. 0.012 0.005 * 
2016xHighPL 0.103 0.041 * 0.094 0.028 *** 0.149 0.047 ** 0.013 0.006 * 
2017xHighPL 0.124 0.048 ** 0.103 0.034 ** 0.157 0.049 ** 0.015 0.006 * 
Covariates Grade 7 -0.036 0.010 * 0.305 0.010***| -0.048 0.018 ** -0.010 0.002 *** 
Grade 8 -0.048 0.011 *** 0.541 0.010 *** 0.072 0.017 *** 0.009 0.002 *** 
Algebra 0.027 0.018 0.217 0.018 ***| -0.065 0.028 * -0.003 0.003 
Other math -0.460 0.202 * 0.885 0.163 *** 0.404 0.343 0.059 0.017 *** 
Multiple math 0.049 0.026 . 0.124 0.024 ** 0.180 0.041 ***} -0.006 0.004 
White -0.140 0.018 *** 0.098 0.011 *** 0.257 0.019 *** 0.011 0.003 *** 
Asian -0.081 0.027 * 0.199 0.016 *** 0.508 0.031 *** 0.024 0.005 *** 
Hispanic -0.148 0.012 *** 0.037 0.008 *** 0.127 0.014 *** 0.006 0.002 * 
Other -0.089 0.026 *** 0.038 0.016 * 0.100 0.027 ***} -0.001 0.005 
Male -0.033 0.007 *** 0.033 0.003 ***) -0.372 0.009***) -0.034 0.002 *** 
Poverty 0.012 0.005 * -0.004 0.003 -0.010 0.005 * -0.002 0.001 * 
Social Status 0.001 0.005 0.003 0.003 -0.009 0.005. -0.001 0.001 
Prior math score 0.114 0.010 *** 0.786 0.006 *** 0.730 0.011 *** 0.048 0.003 *** 


Note: .<.10, *<.05, **<.01, ***<.001. Average achievement is defined as at within 0.5 s.d. of the district mean in 
the prior year on the state assessment. Coefficients come from models that also include school fixed-effects. 
Standard errors were clustered by school. Math practices and NWEA scores were standardized using means and 
standard deviations from 2014. Teacher professional learning (PL) around the CCSS-M was measured in the 2014- 
15 school year, which was the same year all schools were to begin implementing the standards. 
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Table C.3. Results for High Achieving Students with Full Data 


Coefficient 


Estimate S.E. 


Math Practices 


NWEA Score 
Estimate S.E. 


GPA 
Estimate S.E. 


Pass Math 
Estimate 


S.E. 


Intercept -0.092 0.018 ***| -0.084 0.012 *** 2.961 0.016 *** 
2011 (Pre-CCSSM) -0.029 0.034 -0.164 0.032 ***| -0.006 0.002 * 
Year Fixed 2012 (Pre-CCSSM) -0.019 0.033 -0.108 0.032 ***} -0.003 0.002 
Effects 2013 (Pre-CCSSM) 0.098 0.031 ** | -0.082 0.020 *** | -0.047 0.020 * -0.001 0.001 
(Relative to 2015 (Post) 0.070 0.026 ** 0.081 0.012*** | -0.013 0.021 0.001 0.001 
2013) 2016 (Post) 0.104 0.027 ***| 0.143 0.023 *** 0.080 0.026 ** 0.001 0.002 
2017 (Post) 0.048 0.034 0.191 0.025 *** 0.074 0.030 * 0.004 0.001 
201 1xAveragePL (Pre) 0.019 0.048 0.071 0.041. 0.002 0.003 
201 2xAveragePL (Pre) 0.006 0.045 0.061 0.039 0.000 0.003 
Schools with | 2013xAveragePL (Pre) -0.029 0.040 -0.008 0.025 0.032 0.030 0.000 0.002 
Average PL 201 5xAveragePL 0.019 0.035 0.021 0.018 0.018 0.029 0.000 0.002 
2016xAveragePL 0.086 0.038 * 0.037 0.028 -0.019 0.034 0.003 0.002 
2017xAveragePL 0.097 0.046 * 0.016 0.033 0.013 0.038 0.000 0.002 
201 1xHighPL (Pre) -0.053 0.044 0.059 0.041 0.002 0.003 
201 2xHighPL (Pre) -0.028 0.043 0.083 0.040 * 0.000 0.003 
Schools with] = 2013xHighPL (Pre) -0.089 0.040 * 0.018 0.026 0.046 0.030 0.000 0.002 
Extensive PL 2015xHighPL 0.071 0.035 * 0.021 0.021 0.036 0.030 0.001 0.002 
2016xHighPL 0.018 0.037 0.042 0.030 0.049 0.035 0.002 0.002 
201 7xHighPL 0.074 0.046 0.064 0.035 . 0.086 0.038 * 0.000 0.002 
Covariates Grade 7 -0.029 0.010 ** 0.390 0.011 *** 0.010 0.015 -0.002 0.001 ** 
Grade 8 -0.086 0.014***/} 0.645 0.012 *** | -0.037 0.018 * 0.000 0.001 
Algebra -0.005 0.018 0.177 0.014 *** | -0.057 0.019 ** 0.001 0.001 
Other math -0.043 0.077 0.342 0.064 *** 0.360 0.129 ** 0.026 0.018 
Multiple math 0.065 0.023 ** 0.122 0.019 *** 0.108 0.023 ***| -0.001 0.001 
White -0.074 0.016***/ 0.138 0.011 *** 0.225 0.018***| 0.004 0.001 ** 
Asian -0.040 0.017 * 0.202 0.016 *** 0.330 0.022***) 0.006 0.002 *** 
Hispanic -0.081 0.014***} 0.038 0.011 *** 0.117 0.016***| 0.003 0.001 * 
Other -0.059 0.020 * 0.132 0.014 *** 0.160 0.023***| 0.003 0.002 * 
Male -0.026 0.007 ***} 0.052 0.004 *** | -0.287 0.007***| -0.006 0.001 *** 
Poverty 0.005 0.005 -0.014 0.003 ** | -0.023 0.004***} 0.000 0.000 
Social Status 0.002 0.005 0.011 0.004 * 0.006 0.005 0.000 0.001 
Prior math score 0.040 0.006***|} 0.590 0.007 *** 0.429 0.009***| 0.005 0.001 *** 


Note: .<.10, *<.05, **<.01, ***<.001. High achievement is defined as at least 0.5 s.d. above the district mean in the 
prior year on the state assessment. Coefficients come from models that also include school fixed-effects. Standard 
errors were clustered by school. Math practices and NWEA scores were standardized using means and standard 
deviations from 2014. Teacher professional learning (PL) around the CCSS-M was measured in the 2014-15 school 
year, which was the same year all schools were to begin implementing the standards. 
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Professional Learning as a Continuous Variable 

The models were also run with professional learning as a continuous variable, instead of 
separating schools into three groups based on the extensiveness of their professional learning. 
Table C4 shows the results with all students combined, while Table C5 shows the results for 
models separated by prior student achievement. The coefficients that emerge as significant for 
the continuous version are similar to those seen for schools with extensive professional learning, 
although the coefficient on the continuous PL variable predicting NWEA scores is not significant 
until the 2017 year. In the subgroup analyses shown in Table C6, the patterns of which of the 
continuous-PL coefficients are significant is also similar to those observed among schools with 
extensive professional development in the prior tables, with many reaching a level of statistical 


significance for low- and average-achieving students, but not for high-achieving students. 
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Table C4. Professional Learning as a Continuous Variable Predicting Changes in Student 
Outcomes: All Students 


Math Practices NVWVEA Score GPA Pass Math 
Coefficient Estimate S.E. Estimate S.E. Estimate S.E. Estimate S.E. 
Intercept -0.083 0.011 *** -0.151 0.008 *** PFN) (ohana) 0.002 *** 


2011 (Pre-CCSSM) 
2012 (Pre-CCSSM) 
2013 (Pre-CCSSM) 


-0.121 0.016 *** 
-0.072 0.014 *** 
0.017 0.013 


-0.092 0.010 *** 


-0.133 0.017 *** 
-0.058 0.016 *** 
-0.019 0.013 


0.003 *** 
0.003 *** 
0.002 *** 


2015 (Post) 0.121 0.015 *** 0.090 0.008 *** 0.059 0.013 *** 0.002 *** 
2016 (Post) 0.157 0.014 *** 0.158 0.011 *** 0.147 0.015 *** 0.002 *** 
2017 (Post) 0.162 0.017 *** 0.180 0.013 *** 0.154 0.016 *** 0.003 *** 


2011 x PL (Pre) 
2012 x PL (Pre) 
2013 x PL (Pre) 
2015 x PL 
2016 x PL 
2017 x PL 
Grade 7 
Grade 8 
Algebra 
Other math 
Multiple math 
White 
Asian 
Hispanic 
Other 
Male 
Poverty 
Social Status 
Prior math score 
R-square 


Note: .<.10, *<.05, **<.01, ***<.001. Coefficients canie from models that als 


-0.035 0.019 . 
-0.026 0.019 
-0.020 0.015 
0.039 0.017 * 
0.029 0.014 * 
0.037 0.019 . 
-0.038 0.008 *** 
-0.048 0.010 *** 
-0.022 0.015 
-0.127 0.097 
0.056 0.021 ** 
-0.114 0.011 *** 
-0.078 0.015 *** 
-0.133 0.009 *** 
-0.087 0.016 *** 
-0.042 0.005 *** 
0.007 0.004 * 
0.002 0.004 


-0.020 0.010 * 
0.002 0.010 
0.016 0.015 
0.030 0.015 * 
0.319 0.010 *** 
0.580 0.010 *** 
0.222 0.014 *** 
0.553 0.069 *** 
ORI 2 Ol0 22 
0.096 0.010 *** 
0.165 0.013 *** 
0.028 0.008 *** 
0.073 0.012 *** 
0.041 0.003 *** 
-0.011 0.002 *** 
0.003 0.003 


0.010 0.017 
0.008 0.016 
0.000 0.013 
0.033 0.014 * 
0.056 0.016 *** 
0.041 0.017 * 
-0.013 0.014 
0.106 0.014 *** 
-0.124 0.019 *** 
0.167 0.117 
0.159 0.032 *** 
0.230 0.013 *** 
0.369 0.021 *** 
0.111 0.010 *** 
0.127 0.018 *** 
-0.308 0.006 *** 
-0.015 0.003 *** 
-0.002 0.004 


0.004 
0.003 
0.003 
0.002 ** 
0.003 *** 
0.003 *** 


Standard errors were clustered by school. Math practices and NWEA scores were standardized using means and 

standard deviations from 2014. Teacher professional learning (PL) around the CCSS-M was measured in the 2014- 
15 school year, which was the same year all schools were to begin implementing the standards. It was standardized 
around across school means prior to analysis. 
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Table C5. Professional Learning as a Continuous Variable Predicting Changes in Student 
Outcomes: DiD Coefficients for Subgroups of Students by Prior Achievement 


Math Practices NWEA Score GPA Pass Math 
Coefficient Estimate _S.E. -val | Estimate S.E. -val| Estimate S.E. -val | Estimate SE. -val 
2011 x PL (Pre) -0.015 0024 0.53 0.000 0.024 1.00 0.004 0.009 0.69 
2012 x PL (Pre) -0.016 0.023 0.50 0.011 0.024 065 0.001 0.007 0.93 
2013 x PL (Pre) 0.007 0022 0.76 -0.030 0.014 0.03 -0.011 0.019 057 -0.003 0.006 0.57 
Low- 2015 x PL 0.058 0.028 0.04 0.008 0.015 0.61 0.047 0.020 0.02 0.012 0.006 0.04 
Achieving 2016 x PL 0.048 0.023 0.04 0.035 0.018 0.05 0.073 0.022 0.00 0.021 0.007 0.00 
Students 2017 x PL 0.032. 0.029 0.28 0.050. 0.021 0.02 0.055 0.023. 0.02 0.016 0.007 0.03 
2011 x PL (Pre) -0.048 0.025 0.05 0.010 0.023 068 0.002 0.004 0.63 
2012 x PL (Pre) -0.029 0.023 0.20 -0.004 0.020 0.85 -0.006 0.003 0.04 
2013 x PL (Pre) -0.026 0018 0.14 -0.021 0.012 0.08 -0.001 0.018 097 -0.002 0.002 0.47 
Average- 2015 x PL 0.025 0020 0.21 0.007 0.012 0.57 0.027 0.019 016 0.005 0.003 0.05 
Achieving 2016 x PL 0.028 0019 013 0.011 0.016 0.49 0.058 0.021 0.01 0.007 0.003 0.03 
Students 2017 x PL 0.039 0022 0.08 0.044 0.018 0.01 0.043 0.023 006 0.006 0.003 0.08 
2011 x PL (Pre) -0.025 0.022 0.26 0.021 0.019 0.27 0.001 0.001 0.59 
2012 x PL (Pre) -0.026 0.022 0.24 0.017 0.016 030 -0.001 0.001 0.51 
2013 x PL (Pre) -0.031 0.019 0.11 -0.008 0.013 0.53 0.012 0.013 036 0.000 0.001 0.77 
High- 2015 x PL 0.031 0018 0.09 -0.008 0.011 0.48 0.007 0.013 057 0.000 0.001 0.90 
achieving 2016 x PL 0.007 0017 067 0.000 0.015 0.98 0.017 0.016 029 0.001 0.001 0.32 
students 2017 x PL 0.027 0.022 0.21 0.013 0.015 0.40 0.017 0.017. 033 0.000 0.001 0.81 


Note: Coefficients come from models that also include school fixed-effects. Standard errors were clustered by 
school. Math practices and NWEA scores were standardized using means and standard deviations from 2014. 
Teacher professional learning (PL) around the CCSS-M was measured in the 2014-15 school year, which was the 
same year all schools were to begin implementing the standards. It was standardized around across school means 
(mean and s.d. of school means) prior to analysis. 


PARCC Scores as an Outcome 
In 2015, Illinois switched its state test to the PARCC, which was developed in response 


to the CCSS-M. Because there are no scores available prior to standards implementation, we 


cannot conduct DiD analyses to examine changes over time in PARCC scores relative to pre- 


implementation years. Instead, we examine whether schools differed in their average PARCC 


scores in 2015, and in their growth in PARCC scores from 2015 to 2016 and 2017, controlling 


for student covariates, based on their level of PL in 2015. We do not use school fixed effects, as 


these would be estimated only on post-implementation years for schools in each of the PL 


implementation groups, and would be confounded with membership in the implementation 


groups. Instead, we use hierarchical linear models, with students nested within schools (see 


formula below). 
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The models estimate a random effect for each school (oj), which is the average 
difference in PARCC scores for each school, controlling for the variables in the models. Bo; 
captures the school average PARCC score in 2015, adjusted for student covariates. Bi; and B9; 
capture the deviation for each school from their 2015 mean. At the school level, yoo is the 
average PARCC score in 2015 for schools with low levels of PL in 2015, while yo: and yo2 are 
the differences in PARCC scores in 2015 for schools with average and extensive PL, relative to 
those with limited PL, controlling for average school achievement in 2014 on the prior state 
test (the ISAT). Likewise, y11 and y12 are the differences in the deviation from 2015 for schools 
with average and extensive PL, relative to those with limited PL, controlling for average 


school achievement in 2014 on the ISAT, with the 2017 deviation modeled similarly. 


Level-1 Model 


PARCC 4 = B oj + B1*(D16) 4 + B2*(D17) j + Bs*(CGR7) i + Ba*(CGR8) i + Bs*(CALGEBRA) 4 + Boe*(COTH); 
+ Br*(CMULTD 4 + Bs*(CDWHITE) § + Bo*(CDASIAN) ij + Bio*(CDHISP) yj + Bii*(CDOTHER) 4 + 
Bi2*(CDMALE) 4 + Bis *(CSCON) 4 + Bia*(CSSOC) 4 + Bisj*(PRIORM) 4 + rij 


Level-2 Model 
Boj = Yoo + yo *(Ave PL); + Yoo *(High PL); + yo3*(20 14 SchoolAch) ; + Uoj 
Bi = y10 + yr1*(Ave PL); + yi2*(High PL); + yi3*(2014SchoolAch) ; 
B2 = y20 + yo*(Ave PL); + y22*(High PL); + y23*(2014SchoolAch) ; 
Bs = 30 


Bis = Y150 


As shown in Table C6, schools in the three PL groups did not differ in their PARCC 
scores in 2015, controlling for the covariates; neither the average nor extensive PL coefficients 
are significantly different from zero on the intercept (2015). In 2016, schools with limited PL did 
not show significant improvement in PARCC scores relative to their scores in 2015 (insignificant 
coefficient of 0.014 s.d.). Schools with average and extensive PL showed significantly larger 


improvements in that year than schools with limited — PL (coefficients of 0.025 and 0.038, 


11 


IMPROVEMENTS IN MATH INSTRUCTION AND ACHIEVEMENT WITH CCSS-M 


respectively; total change of 0.039 and 0.052, respectively). In 2017, schools with extensive 
professional learning in 2015 continued to show significantly higher gains in PARCC scores than 
schools with limited PL. However, in 2017 schools with average PL no longer showed higher 
PARCC scores than schools with limited PL, which differs from NWEA scores. It is not clear 
whether the difference results because of the outcome being measured (PARCC vs NWEA), the 
lack of pre-implementation data, different methodology, or different base year. But the general 
pattern is similar—significant differences did not appear until 2016, when they are seen among 
both schools with average and extensive PL, and they remained significant among schools with 


extensive PL in 2017, at about the same level as in 2016. 
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Table C6. Models Predicting PARCC Scores 


Fixed Effects Coeff s.e. 
Intercept (2015, Limited PL) -0.014 0.015 
Average PL (relative to Limited PL) 0.012 0.018 
Extensive PL (relative to Limited PL) 0.000 0.018 
2014 mean achievement 0.145 0.014 *** 
Year 2016 (relative to 2015) 0.014 0.010 
Average PL (relative to Limited PL) 0.025 0.012 * 
Extensive PL (relative to Limited PL) 0.038 0.012 ** 
2014 mean achievement -0.006 0.010 
Year 2017 (relative to 2015) -0.014 0.009 
Average PL (relative to Limited PL) 0.000 0.012 
Extensive PL (relative to Limited PL) 0.035 0.012 ** 
2014 mean achievement 0.050 0.010 ** 
Grade 7 0.061 0.004 *** 
Grade 8 -0.112 0.005 *** 
Algebra 0.414 0.008 *** 
Other math 0.652 0.064 *** 
Multiple math 0.193 0.012 *** 
White 0.168 0.009 *** 
Asian 0.256 0.012 *** 
Hispanic 0.074 0.007 *** 
Other 0.104 0.017 *** 
Male -0.049 0.004 *** 
Poverty -0.008 0.003 *** 
Social Status -0.003 0.003 
Prior math score 0.752 0.002 *** 
Random Components Variance in SD 
Intercept, uo 0.014 0.119 *** 
Residual, r 0.352 0.594 *** 


*p<.05, **p<.01, ***p<.001. PARCC scores are standardized based on the 2015 mean and 
standard deviation across students. Student-level prior math score is the prior state test ISAT) 
for students in 2015 (2014 ISAT scores) and the PARCC in later years, each standardized within 
year and grade. School-level mean achievement is based on the 2014 ISAT. Analysis was 
conducted through hierarchical linear models with students nested within schools. 
Including Teacher Fixed Effects 

An additional specification adds teacher fixed effects to the models. Adding teacher fixed 


effects could address any concerns that there were changes in the teacher workforce in post- 


implementation years that account for changing instruction in schools with more professional 
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development around the standards. However, it also can confound teacher and school effects, as 
some teachers only have observations in a small number of years, and many do not have 
observations in both pre-implementation and post-implementation years. To the extent that the 
middle grade teachers in a school in post-implementation years were different than those in the 
school in pre-policy years, it becomes difficult to attribute changes in practices and outcomes in 
the school to teachers versus school factors. 

Table C7 shows that 40% of teachers in the sample only are observed in one of the seven 
years. This occurs not only because teachers may leave or be hired in different schools, but also 
because teachers move in and out of teaching in grades 6-8 (most CPS elementary schools span 
grades K-8). While this is a large percentage of teachers in the sample, it is only eight percent of 
the observations; the more years a teacher is present the more observations they can have in the 
sample, and long-term teachers tend to have more students in a given year than teachers who are 
observed just once. Only 36 percent of teachers taught students in the sample in both pre- 
implementation years (2014 and earlier) and post-implementation years (2015 and later). At the 
same time, 73 percent of the observations were attached to teachers that had observations in at 
least one pre-implementation and one post-implementation year. The further the observation was 
from 2014/2015, the less likely the teacher was observed in both years. The coefficients and 
standard errors should be interpreted cautiously not only because of the issues described above, 
but also because it is difficult to interpret the coefficients. The DiD coefficients are framed in 
reference to the 2014 year, and more than a quarter of all teachers did not teach in 2014, or any 
pre-policy year, making interpretation of coefficients and standard errors ambiguous. 

One additional difference from the model without teacher fixed effects (Table C4) is that 


we only could include one teacher record per observation. In the prior analysis, we averaged 
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together grades and pass rates for students who took multiple math courses in a year, but it would 
not make sense to average teacher ID numbers. For this analysis, we deleted observations that 
were takenin math classes other than standard mathematics or algebra, so that we would use 
the record from a student’s primary math class for the analysis. We also deleted any observations 
in which multiple teacher IDs were assigned to the same class (e.g., classes that were co-taught). 
These two restrictions only reduced the number of cases by 2.5 percent of the total. 

Results from the models are shown in Table C8; they can be compared to those in Table 
C4 which is the same specification but without teacher fixed effects. The sizes of the DiD 
coefficients are similar in the two tables, although fewer of the coefficients are significant. The 
GPA and passing coefficients representing 2016 and 2017 remain significant in this 
specification, as well as the math practices measure in 2017. Strong similarity in the size of the 
coefficients in Table C4 provides evidence that the findings on grades and practices were not 
simply a result of changes in the teacher workforce. Changes in the significance levels in the 
model with teacher fixed effects could result from the lack of within-teacher variation in each 
school year, which is the major component of the DiD estimates. 

Consideration of Student Fixed Effects. Readers may be interested in model 
specifications that include student fixed-effects, which would provide estimates of how 
individual students’ outcomes changed relative to themselves in other years that they were in 
grades 6, 7 or 8. Because we only study the middle grades, even in the complete data set that 
includes all observations, only 27 percent of observations are based on students who have data in 
both pre-implementation and post-implementation years (see Table C9). The observations that 
span pre- and post-implementation periods are more likely to occur in the years immediately 


before and after implementation (2014 and 2015), so that estimates for 2016 and 2017 are based 
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on small numbers of students. Only the small percentage of students held back in grade have 


observations for both 2014 and 2017. Because students must have at least two observations to be 


included in the analysis, the seventh-grade year becomes over-weighted relative to other years. 


For these reasons, adding student fixed-effects to the models makes the year and DiD 


coefficients uninterpretable. 


Table C7. Number of years each teacher is observed in the sample 


Number of years 
teacher is Number of 
observed in the Number of Percent of Student Percent of 
sample Teachers Teachers Observations Observations 
1 year 1301 40% 24,481 8% 
2 years 634 19% 32,265 11% 
3 years 372 11% 32,227 11% 
4 years 305 9% 38,962 13% 
5 years 226 7% 41,488 14% 
6 years 178 5% 48,360 16% 
7 years 237 7% 86,865 29% 
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Table C8. Models with Teacher Fixed Effects and Continuous Measure of Professional 


Learning 
Math Practices NWEA Score GPA Pass Math 
Estim 
Estimate S.E. ate S.E. Estimate S.E. Estimate _S.E. 
Intercept -0.002 0.01 -0.026 0.01 ** 2.532 0.01 *** 0.955 0.002 *** 
2011 (Pre-CCSSM) -0.099 0.02 ** -0.094 0.02 ** -0.025 0.004 *** 
2012 (Pre-CCSSM) -0.061 0.02 ** -0.038 0.01 * -0.014 0.003 *** 
2013 (Pre-CCSSM) 0.022 0.01 -0.095 0.01 ** -0.015 0.01 -0.009 0.002 *** 
2015 (Post) 0.113 0.01 ** | 0.092 0.01 ** 0.043 0.01 ™ 0.006 0.002 *™ 
2016 (Post) 0.143 0.02 ** | 0.169 0.01 ** 0.126 0.01 ** 0.017 0.003 *** 
2017 (Post) 0.148 0.02 ** | 0.194 0.02 ** 0.136 0.02 *** 0.017. 0.003 *** 
2011 x PL (Pre) -0.044 0.02 0.000 0.02 -0.005 0.004 
2012 x PL (Pre) -0.032 0.02 -0.001 0.02 -0.006 0.003 
2013 x PL (Pre) -0.017 0.02 -0.030 0.01 ™ -0.008 0.01 -0.002 0.003 
2015 x PL 0.028 0.02 0.000 0.01 0.022 0.02 0.004 0.003 
2016 x PL 0.024 0.02 0.017 0.02 0.046 0.02 * 0.008 0.003 *™ 
2017 x PL 0.040 0.02 * 0.024 0.02 0.061 0.02 ™ 0.009 0.004 * 
Grade 7 -0.026 0.01 ™ 0.339 0.01 ** -0.010 0.01 0.001 0.002 
Grade 8 -0.040 0.01 ™ 0.584 0.01 ** 0.124 0.01 ** 0.026 0.003 *** 
Algebra -0.022 0.01 0.221 0.01 ** -0.134 0.02 ** -0.021 0.002 ** 
Multiple math 0.060 0.02 * 0.139 0.02 ** 0.160 0.03 *** -0.002 0.004 
White 0.115 0.01 *** | 0.097 0.01 ** 0.207 0.01 *** 0.003 0.003 
Asian -0.076 0.01 *** | 0.156 0.01 ** 0.356 0.02 *** 0.005 0.004 
Hispanic 0.127 0.01 *** | 0.031 0.01 ** 0.103 0.01 ** 0.007 0.002 *™ 
Other -0.078 0.02 *** | 0.075 0.01 ** 0.108 0.02 ** 0.000 0.003 
Male -0.041 0.01 *** | 0.051 o ™ -0.323 0.01 *** -0.034 0.002 *** 
Poverty 0.008 Oo * -0.010 Q “* -0.017 Q -0.001 7E-04 
Social Status 0.004 0 0.003 0 -0.004 0 -0.001 8E-04 
Prior math score 0.088 o ** | 0.729 Q 0.617 0.01 *** 0.043 0.002 *** 
R-square 0.089 0.746 0.453 0.127 


*p<.05, **p<.01, ***p<.001. Coefficients come from models that include school and teacher fixed-effects. Standard 
errors were clustered by school. Math practices and NWEA scores were standardized using means and standard 
deviations from 2014. Teacher professional learning (PL) around the CCSS-M was measured in the 2014-15 school 
year, which was the same year all schools were to begin implementing the standards. It was standardized around 
across school means (mean and s.d. of school means) prior to analysis. 


Table C9. Number of years each student is observed in the full sample 


Number of 
years student is 
observed in the 

sample 
1 time 
2 times 
3 times 
4 times 
5 times 


Number Percent 
of of 
Students Students 
75,416 13.8% 
135,081 24.7% 
329,775 60.4% 
6,137 1.1% 

15 0.0% 


Number of Percent of 
Observations Observations 
75,416 5.5% 
270,162 19.9% 
989,325 72.8% 
24,548 1.8% 
75 0.0% 


Percent with 
observations 


post- 
implementation 
0% 
8% 
40% 
79% 
100% 


spanning pre- and 


Percent of total 
observations 
spanning pre- and 
post- 
implementation 
0% 

2% 

24% 

1% 

0% 
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